What is claimed is: 



1 1 . A method for robust multi-pass variable bit rate video encoding, the method 

2 comprising: 

3 an encoding manager performing a first-pass encoding of a video sequence; 

4 the encoding manager collecting data concerning the video sequence during the 

5 first-pass encoding; 

6 the encoding manager utilizing collected data to construct a data analysis model 

7 concerning the video sequence, the data analysis model comprising at least 

8 a frame model concerning each frame of the video sequence, and a 

9 sequence model concerning the video sequence as a whole; 

10 the encoding manager utilizing the data analysis model to generate a rate profile 

1 1 for the video sequence, the rate profile complying with a bit budget for the 

12 video sequence, avoiding buffer underflow for each frame of the video 

13 sequence, distinguishing between easy and hard segments of the video 

14 sequence, and allocating bits to segments as a function of segment 

15 complexity; and 

16 the encoding manager utilizing the rate profile to perform a second-pass encoding 

17 of the video sequence. 

1 2. The method of claim 1 wherein the encoding manager performing the first-pass 

2 encoding of the video sequence comprises the encoding manager performing a step from a group 

3 of steps consisting of: 

4 encoding the video sequence with a constant Q, without rate control; 
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5 encoding the video sequence utilizing one pass variable bit rate encoding, thereby 

6 attempting to achieve a target bit rate; and 

7 encoding the video sequence utilizing one pass constant bit rate encoding. 

1 3. The method of claim 1 wherein the encoding manager collecting data concerning the 

2 video sequence during the first-pass encoding further comprises: 

3 the encoding manager collecting data to be used to construct a data analysis 

4 model concerning the video sequence, the collected data comprising at 

5 least one data point concerning each frame of the sequence from a list of 

6 data points consisting of: 

7 a picture type; 

8 a bit total; 

9 a DCT bit total; 

10 an average mquant; 

11 an average activity; 

12 a scene change indicator; 

13 a fade indicator; 

14 a still frame indicator; and 

15 a transition indicator. 

1 4. The method of claim 1 wherein the encoding manager utilizing collected data to 

2 construct a data analysis model concerning the video sequence farther comprises: 

3 the encoding manager utilizing collected data to construct a frame model 

4 concerning each frame of the video sequence, the frame model comprising 
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a formula expressing a mathematical relationship between frame bit rate, 
frame complexity, frame Q and frame bit overhead for each frame of the 
video sequence. 



1 5. The method of claim 4 further comprising: 

2 the encoding manager utilizing collected data to calculate complexity and bit 

3 overhead for each frame of the video sequence, and using the calculated 

4 values in the frame model. 

1 6. The method of claim 1 wherein the encoding manager utilizing collected data to 

2 construct a data analysis model concerning the video sequence and further comprises: 

3 the encoding manager utilizing collected data to construct a sequence model 

4 concerning the video sequence as a whole, the sequence model identifying 

5 transitions in the video sequence. 

1 7. The method of claim 1 wherein the encoding manager utilizing the data analysis 

2 model to generate a rate profile for the video sequence further comprises: 

3 the encoding manager calculating an initial Q for the video sequence as a function 

4 of a sum of frame complexity of the frames of the sequence, a sum of bit 

5 overhead of the frames of the sequence, and the bit budget for the video 

6 sequence. 

1 8. The method of claim 7 further comprising: 

2 the encoding manager calculating an initial bit rate profile for the video sequence 

3 by, for each frame of the video sequence, calculating a bit rate for that 
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frame as a function of the calculated initial Q, the complexity of the frame 
and the bit overhead of the frame. 

9. The method of claim 8 further comprising: 

the encoding manager adjusting the calculated a bit rate for at least one frame 
based on at least one factor from a group of factors consisting of: 
the calculated bit rate being less than a minimum number of bits for a 
frame; 

the calculated bit rate being at least as few bits as the bit overhead for the 
frame; and 

the frame being a transition frame in the video sequence. 

10. The method of claim 8 further comprising: 

the encoding manager simulating decoding of at least a portion of the video 

sequence according to an initial rate profile; 
the encoding manager determining whether any frames underflow the buffer; 
the encoding manager determining whether any frames overflow the buffer; 
responsive to a segment of the video sequence transitioning the buffer from 

overflow to underflow, the encoding manager classifying that segment as 

hard; 

the encoding manager calculating an updated Q for each hard segment, so as to 
avoid the underflow of that hard segment; and 
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1 1 the encoding manager calculating an updated Q for the video sequence absent any 

12 hard segments, based on the number of bits added to the bit budget as a 

13 result of calculating an updated Q for each hard segment; and 

14 the encoding manager reformulating the initial rate profile based on the updated 

15 Qs for each hard segment and the updated Q for the video sequence absent 

16 any hard segments. , 

1 11. The method of claim 10 further comprising: 

2 the encoding manager repeating the steps of claim 10, until a condition occurs 

3 from a group of conditions consisting of: 

4 the encoding manager simulating decoding of the video sequence 

5 according to an initial rate profile such that the video sequence 

6 contains no hard segments; and 

7 the encoding manager simulating decoding of the video sequence a 

8 maximum number of times; 

9 and; 

10 the encoding manager classifying the rate profile last used to simulate decoding of 

1 1 the video sequence as the generated rate profile for the video sequence. 

1 12. The method of claim 7 or claim 10 further comprising: 

2 the encoding manager calculating a separate Q for I frames, P frames and B 

3 frames. 

1 13. The method of claim 1 wherein the encoding manager utilizing the rate profile to 



2 perform a second-pass encoding of the video sequence further comprises: 
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3 the encoding manager encoding each frame of the video sequence according to a 

4 Q calculated for that frame by the encoding manager during rate profile 

5 generation; and 

6 the encoding manager determining, for each frame of the video sequence, whether 

7 a bit rate for a frame encoded according to the calculated Q is within a 

8 margin of error of a bit rate calculated for that frame by the encoding 

9 manager during rate profile generation. 

1 14. The method of claim 13 further comprising: 

2 responsive to determining that a bit rate for the frame encoded according to the 

3 calculated Q is within a margin of error of the bit rate calculated for that 

4 frame during rate profile generation, the encoding manager accepting that 

5 encoding for that frame. 

1 15. The method of claim 13 further comprising: 

2 responsive to determining that a bit rate for a frame encoded according to the 

3 calculated Q is not within a margin of error of the bit rate calculated for 

4 that frame during rate profile generation, the encoding manager: 

5 for each macroblock of that frame, constructing a macroblock model 

6 comprising a formula expressing a mathematical relationship 

7 between complexity, bit overhead, and updated bit rate, and an 

8 updated Q and for that macroblock; 

9 encoding each macroblock according to its corresponding macroblock 
10 model; and 
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1 1 collecting modeling data concerning each macroblock during the encoding 

12 thereof. 

1 16. The method of claim 15 further comprising: 

2 for each macroblock of the frame, the encoding manager calculating an updated Q 

3 for that macroblock as a function of base Q for the macroblocks of the 

4 frame remaining to be encoded and the activity mask for that macroblock; 

5 for each macroblock of the frame, the encoding manager calculating an updated 

6 bit rate for the macroblock, based on the updated Q, and the complexity 

7 and bit overhead of the macroblock according to the last encoding thereof; 

8 and 

9 for each macroblock of the frame, after encoding that macroblock, the encoding 

10 manager updating, according to the encoding of that macroblock, base Q 

1 1 for the macroblocks of the frame remaining to be encoded. 

1 17. The method of claim 15 further comprising: 

2 the encoding manager repeating the steps of claim 1 5, until a condition occurs 

3 from a group of conditions consisting of: 

4 the encoding manager encoding each macroblock of the frame such that a 

5 bit rate for the frame as encoded at a macroblock level is within a 

6 margin of error of the bit rate calculated for that frame by the 

7 encoding manager during rate profile generation; and 

8 the encoding manager encoding the frame at a macroblock level a 

9 maximum number of times; 
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10 and; 

1 1 the encoding manager accepting the last encoding of the frame at a macroblock 

1 2 level as the encoding for that frame. 

1 18. The method of claim 14 or claim 17 further comprising: 

2 the encoding manager determining whether the encoding of the frame causes 

3 underflow; and 

4 responsive to determining that the encoding of the frame causes underflow, the 

5 encoding manager adjusting the bit rate of the frame so as to eliminate the 

6 underflow. 

1 19. The method of claim 1 wherein the encoding manager utilizing the rate profile to 

2 perform a second-pass encoding of the video sequence further comprises: 

3 for each frame of the video sequence, the encoding manager performing the 

4 following steps: 

5 refining at least one model parameter concerning that frame from a group 

6 of model parameters consisting of: 

7 bit rate; and 

8 complexity; 

9 updating the model for that frame based on at least one refined model 

10 parameter; 

1 1 calculating an optimized Q for that frame based on the updated model; and 

12 encoding the frame according to the optimized Q. 

1 20. The method of claim 19 further comprising: 
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for each frame of the video sequence, the encoding manager ensuring that the 

optimized Q for that frame conforms to parameters concerning the video 
sequence. 



1 21. The method of claim 19 further comprising: 

2 for each frame of the video sequence, the encoding manager determining whether 

3 that frame as encoded according to an optimized'Q results in buffer 

4 underflow. 

1 22. The method of claim 21 further comprising: 

2 responsive to determining that encoding a frame according to an optimized Q 

3 results in buffer underflow, the encoding manager repeating the steps of 

4 claim 19, until a condition occurs from a group of conditions consisting 

5 of: 

6 the encoding manager determining that encoding a frame according to an 

7 optimized Q does not result in buffer underflow; and 

8 the encoding manager encoding the frame according to an optimized Q a 

9 maximum number of times; 

10 and; 

1 1 the encoding manager accepting the last encoding of the frame as the encoding for 

12 that frame. 

1 23. The method of claim 21 further comprising: 

2 the encoding manager determining that encoding a frame according to its 

3 optimized Q does not result in buffer underflow; and 
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the encoding manager accepting the encoding of the frame according to its 
optimized Q as the encoding for that frame. 



1 24. A system for robust multi-pass variable bit rate video encoding, the system 

2 comprising: 

3 a software portion for performing a first-pass encoding of a video sequence; 

4 a software portion for collecting data concerning the video sequence during the 

5 first-pass encoding; 

6 a software portion for utilizing collected data to construct a data analysis model 

7 concerning the video sequence, the data analysis model comprising at least 

8 a frame model concerning each frame of the video sequence, and a 

9 sequence model concerning the video sequence as a whole; 

10 a software portion for utilizing the data analysis model to generate a rate profile 

1 1 for the video sequence, the rate profile complying with a bit budget for the 

12 video sequence, avoiding buffer underflow for each frame of the video 

1 3 sequence, distinguishing between easy and hard segments of the video 

14 sequence, and allocating bits to segments as a function of segment 

1 5 complexity; and 

16 a software portion for utilizing the rate profile to perform a second-pass encoding 

1 7 of the video sequence. 

1 25. The system of claim 24 further comprising: 
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2 a software portion for encoding each frame of the video sequence according to a 

3 Q calculated for that frame by the encoding manager during rate profile 

4 generation; and 

5 a software portion for determining, for each frame of the video sequence, whether 

6 a bit rate for a frame encoded according to the calculated Q is within a 

7 margin of error of a bit rate calculated for that frame by the encoding 

8 manager during rate profile generation. 

1 26. The system of claim 24 further comprising: 

2 a software portion for, for each frame of the video sequence: 

3 refining at least one model parameter concerning that frame from a group 

4 of model parameters consisting of: 

5 bit rate; and 

6 complexity; 

7 updating the model for that frame based on at least one refined model 

8 parameter; 

9 calculating an optimized Q for that frame based on the updated model; and 
10 encoding the frame according to the optimized Q. 

1 27. A system for robust multi-pass variable bit rate video encoding, the system 

2 comprising: 

3 means for performing a first-pass encoding of a video sequence; 

4 means for collecting data concerning the video sequence during the first-pass 

5 encoding; 
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6 means for utilizing collected data to construct a data analysis model concerning 

7 the video sequence, the data analysis model comprising at least a frame 

8 model concerning each frame of the video sequence, and a sequence 

9 model concerning the video sequence as a whole; 

10 means for utilizing the data analysis model to generate a rate profile for the video 

1 1 sequence, the rate profile complying with a bit budget for the video 

12 sequence, avoiding buffer underflow for each frame of the video sequence, 

13 distinguishing between easy and hard segments of the video sequence, and 

14 allocating bits to segments as a function of segment complexity; and 

15 means for utilizing the rate profile to perform a second-pass encoding of the video 

16 sequence. 

1 28. The system of claim 27 further comprising: 

2 means for encoding each frame of the video sequence according to a Q calculated 

3 for that frame by the encoding manager during rate profile generation; and 

4 means for determining, for each frame of the video sequence, whether a bit rate 

5 for a frame encoded according to the calculated Q is within a margin of 

6 error of a bit rate calculated for that frame by the encoding manager 

7 during rate profile generation. 

1 29. The system of claim 27 further comprising: 

2 means for, for each frame of the video sequence: 

3 refining at least one model parameter concerning that frame from a group 

4 of model parameters consisting of: 
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5 bit rate; and 

6 complexity; 

7 updating the model for that frame based on at least one refined model 

8 parameter; 

9 calculating an optimized Q for that frame based on the updated model; and 
10 encoding the frame according to the optimized Q. 

1 30. A computer readable medium containing a computer program product for robust 

2 multi-pass variable bit rate video encoding, the computer program product comprising: 

3 program code for performing a first-pass encoding of a video sequence; 

4 program code for collecting data concerning the video sequence during the first- 

5 pass encoding; 

6 program code for utilizing collected data to construct a data analysis model 

7 concerning the video sequence, the data analysis model comprising at least 

8 a frame model concerning each frame of the video sequence, and a 

9 sequence model concerning the video sequence as a whole; 

10 program code for utilizing the data analysis model to generate a rate profile for 

1 1 the video sequence, the rate profile complying with a bit budget for the 

12 video sequence, avoiding buffer underflow for each frame of the video 

13 sequence, distinguishing between easy and hard segments of the video 

14 sequence, and allocating bits to segments as a function of segment 

1 5 complexity; and 

16 program code for utilizing the rate profile to perform a second-pass encoding of 

17 the video sequence. 
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3 1 . The computer program product of claim 30 further comprising: 

program code for encoding each frame of the video sequence according to a Q 
calculated for that frame by the encoding manager during rate profile 
generation; and 

program code for determining, for each frame of the video sequence, whether a 
bit rate for a frame encoded according to the calculated Q is within a 
margin of error of a bit rate calculated for that frame by the encoding 
manager during rate profile generation. 

32. The computer program product of claim 30 further comprising: 

program code for, for each frame of the video sequence: 

refining at least one model parameter concerning that frame from a group 
of model parameters consisting of: 
bit rate; and 
complexity; 

updating the model for that frame based on at least one refined model 
parameter; 

calculating an optimized Q for that frame based on the updated model; and 
encoding the frame according to the optimized Q. 
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